215 research outputs found
Training Graph Neural Networks on Growing Stochastic Graphs
Graph Neural Networks (GNNs) rely on graph convolutions to exploit meaningful
patterns in networked data. Based on matrix multiplications, convolutions incur
in high computational costs leading to scalability limitations in practice. To
overcome these limitations, proposed methods rely on training GNNs in smaller
number of nodes, and then transferring the GNN to larger graphs. Even though
these methods are able to bound the difference between the output of the GNN
with different number of nodes, they do not provide guarantees against the
optimal GNN on the very large graph. In this paper, we propose to learn GNNs on
very large graphs by leveraging the limit object of a sequence of growing
graphs, the graphon. We propose to grow the size of the graph as we train, and
we show that our proposed methodology -- learning by transference -- converges
to a neighborhood of a first order stationary point on the graphon data. A
numerical experiment validates our proposed approach
Geometric Graph Filters and Neural Networks: Limit Properties and Discriminability Trade-offs
This paper studies the relationship between a graph neural network (GNN) and
a manifold neural network (MNN) when the graph is constructed from a set of
points sampled from the manifold, thus encoding geometric information. We
consider convolutional MNNs and GNNs where the manifold and the graph
convolutions are respectively defined in terms of the Laplace-Beltrami operator
and the graph Laplacian. Using the appropriate kernels, we analyze both dense
and moderately sparse graphs. We prove non-asymptotic error bounds showing that
convolutional filters and neural networks on these graphs converge to
convolutional filters and neural networks on the continuous manifold. As a
byproduct of this analysis, we observe an important trade-off between the
discriminability of graph filters and their ability to approximate the desired
behavior of manifold filters. We then discuss how this trade-off is ameliorated
in neural networks due to the frequency mixing property of nonlinearities. We
further derive a transferability corollary for geometric graphs sampled from
the same manifold. We validate our results numerically on a navigation control
problem and a point cloud classification task.Comment: 16 pages, 6 figures, 3 table
A Spectral Analysis of Graph Neural Networks on Dense and Sparse Graphs
In this work we propose a random graph model that can produce graphs at
different levels of sparsity. We analyze how sparsity affects the graph
spectra, and thus the performance of graph neural networks (GNNs) in node
classification on dense and sparse graphs. We compare GNNs with spectral
methods known to provide consistent estimators for community detection on dense
graphs, a closely related task. We show that GNNs can outperform spectral
methods on sparse graphs, and illustrate these results with numerical examples
on both synthetic and real graphs.Comment: Extended version of ICASSP 2024 submissio
Increase and Conquer: Training Graph Neural Networks on Growing Graphs
Graph neural networks (GNNs) use graph convolutions to exploit network
invariances and learn meaningful features from network data. However, on
large-scale graphs convolutions incur in high computational cost, leading to
scalability limitations. Leveraging the graphon -- the limit object of a graph
-- in this paper we consider the problem of learning a graphon neural network
(WNN) -- the limit object of a GNN -- by training GNNs on graphs sampled
Bernoulli from the graphon. Under smoothness conditions, we show that: (i) the
expected distance between the learning steps on the GNN and on the WNN
decreases asymptotically with the size of the graph, and (ii) when training on
a sequence of growing graphs, gradient descent follows the learning direction
of the WNN. Inspired by these results, we propose a novel algorithm to learn
GNNs on large-scale graphs that, starting from a moderate number of nodes,
successively increases the size of the graph during training. This algorithm is
benchmarked on both a recommendation system and a decentralized control problem
where it is shown to retain comparable performance, to its large-scale
counterpart, at a reduced computational cost
Convolutional Filtering on Sampled Manifolds
The increasing availability of geometric data has motivated the need for
information processing over non-Euclidean domains modeled as manifolds. The
building block for information processing architectures with desirable
theoretical properties such as invariance and stability is convolutional
filtering. Manifold convolutional filters are defined from the manifold
diffusion sequence, constructed by successive applications of the
Laplace-Beltrami operator to manifold signals. However, the continuous manifold
model can only be accessed by sampling discrete points and building an
approximate graph model from the sampled manifold. Effective linear information
processing on the manifold requires quantifying the error incurred when
approximating manifold convolutions with graph convolutions. In this paper, we
derive a non-asymptotic error bound for this approximation, showing that
convolutional filtering on the sampled manifold converges to continuous
manifold filtering. Our findings are further demonstrated empirically on a
problem of navigation control.Comment: 7 pages, 4 figures, submitted to ICASSP 202
- …